Profiling Python Code

Documentation for Python's standard profilers can be found here: https://docs.python.org/3.5/library/profile.html

Running the Python Code under the profiler

To run your Python using the profiler run:

python -m cProfile -o yourscript.profile yourscript.py

replacing <yourscript> with the name of your script.

Exploring the profile output

To then explore the output of the profiler use pstats as so:

python -m pstats yourscript.profile

Using pstats

Useful pstats commands are:

• help - to get help
• strip - to strip paths from filenames
• sort <key> - to sort the results by the key
• stats <n> - to show the top n results
• callers <pattern> - to show what called the pattern (where pattern could be a function name)
• quit - to quit pstats

A typical first step would be to strip paths from filenames:

strip

then sort by time:

sort time

and then see which functions are the top 10 by time spent:

stats 10

Visualisation of results

https://github.com/jrfonseca/gprof2dot

You will need Graphviz:

Windows users can install graphviz as follows

• Install Graphviz from http://www.graphviz.org/Download_windows.php
• Add C:\Program Files (x86)\Graphviz2.38\bin to the PATH
• Run "pip install graphviz" on the command line

Useful Videos on Profiling

• https://www.youtube.com/watch?v=bectZn_yNwg

• Writing faster Python: https://www.youtube.com/watch?v=YjHsOrOOSuI

Useful References On Performance

https://wiki.python.org/moin/PythonSpeed/PerformanceTips